Skip to content

Docs rewrite 1/5: README, landing page, install, getting started - #185

Merged
EiffL merged 5 commits into
clean_rebuildfrom
docs-user-guide
Aug 21, 2026
Merged

Docs rewrite 1/5: README, landing page, install, getting started#185
EiffL merged 5 commits into
clean_rebuildfrom
docs-user-guide

Conversation

@EiffL

@EiffL EiffL commented Aug 20, 2026

Copy link
Copy Markdown
Member

First pass of the post-rebuild documentation rewrite — the entry points: README.md, docs/index.md, docs/user/{index,install,getting-started}.md. Style deliberately follows the previous versions (tabs, admonitions, numbered steps, the "what just happened" recap); the content is brought to the rebuilt CLI.

What changed

  • README: quick start shows the real loop (init → spec + scripts → uv add → commit → materialize); the Capabilities section replaces lc verify / lc export wrroc / Snakemake with what exists — multiverse materialization, provenance by construction (manifests + run records, annex-carried bytes), locked sandboxed execution, derived containerized mode + SLURM, and the license-derived RO-Crate view.
  • docs/index.md: light touch — drops "skills" from the alpha notice, updates the lightcone-cli card's description.
  • user/index.md: quick start updated (commit before materialize, with a sentence on why the commit isn't ceremony); covered-pages list wording refreshed.
  • user/install.md: rewritten. Prerequisites are uv + git; git-annex ships as a wheel with the install (with the platform floor stated honestly, and WSL for Windows); a new "tell git who you are" step (materialize checks identity up front); container runtime section reflects detection (podman-hpc → podman → docker) and the absence of any config file — ~/.lightcone/config.yaml no longer exists. Upgrade section notes an engine upgrade never invalidates results.
  • user/getting-started.md: new worked example — a line fit over a synthetic dataset with one methodological decision (keep vs clip outliers), swept across baseline and robust universes, ending with the license declaration and the auto-committed ro-crate-metadata.json. It walks through the scaffold, declared inputs and the annex, the dependency contract and placeholders, uv add, the clean-tree rule, real lc materialize / lc status output, incremental rebuilds, and the fresh-clone story.

Verification

  • Every command block in the walkthrough was executed against the current lc before landing; the quoted console output is captured from that run (slope numbers included).
  • The fresh-clone claim was tested: a bare clone refuses, lc init + lc materialize reports up to date with no annex content fetched — the guide says exactly that.
  • zensical build passes; the two remaining link warnings are in frozen pages (api/snakefile.md, hpc/targets.md) that later passes replace.

Next passes: (2) core concepts + cluster/troubleshooting/glossary, (3) CLI + JSON reference, (4) developer corner, (5) nav + workflow refresh.

Two cosmetic engine warts observed while verifying, deliberately not touched here: Rich markup swallows the literal [project] in the license hint (renders as "no .license in pyproject.toml"), and a stray Dask teardown TimeoutError line can appear on stderr after a run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

EiffL and others added 3 commits August 20, 2026 21:59
First pass of the post-rebuild documentation rewrite, covering the
entry points: the README, the docs landing page, the user-guide
welcome, install, and a new getting-started walkthrough.

Every claim tracks the rebuilt CLI: `lc materialize` is the build verb,
states are current/behind/stale, the environment is the project's own
lock managed with `uv add`, builds start from a clean tree and commit
every output with its manifest and run record, and a declared license
turns on the RO-Crate publication view. The install page states the
real prerequisites (uv + git; git-annex arrives as a wheel and sets the
platform floor), the git-identity requirement, and the
detected-never-configured container runtime — the old global config
file is gone.

The walkthrough is a new worked example — a line fit sweeping an
outlier-handling decision across two universes, ending published as a
crate — and every command block was executed against the current `lc`
before landing, including the fresh-clone story (`lc init` rebuilds the
.venv and the annex; the up-to-date verdict needs no annex content).
Console outputs quoted in the guide are captured from the real run.

Style follows the previous docs: tabs, admonitions, numbered steps,
"a few things to notice" asides, and a "what just happened" recap.
The remaining user-guide pages (cluster, troubleshooting, glossary),
the reference, and the developer corner land in the next passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

@EiffL EiffL left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok not bad, made a few comments

Comment thread docs/user/install.md

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need to mention git-annex nor astra-tools, remove these mentions

Comment thread docs/user/getting-started.md Outdated
is downstream of this file. The boilerplate from `lc init` has one example
output and an example decision — enough to run `lc run` and see something
materialize, but not yet a real analysis.
- **The project is a git repository, and that's load-bearing.** Every

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **The project is a git repository, and that's load-bearing.** Every
- **The project is a git repository.** Every

EiffL and others added 2 commits August 21, 2026 10:13
Review notes: the install page no longer names git-annex or astra-tools
— they are implementation details the install command takes care of —
and the getting-started git bullet drops its aside.

And the rendering bug the docs run surfaced: engine prose interpolated
into rich markup strings ate literal brackets, so the license hint
printed "no .license in pyproject.toml" with the one word that names
the fix swallowed as a style tag. All three rendering sites (init,
materialize, status) now escape warnings, whys, item names and paths —
data, never markup. Task names stay bare: ASTRA ids cannot hold a
bracket. Regression test pins the license hint's brackets end to end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx
@EiffL
EiffL merged commit bd68cff into clean_rebuild Aug 21, 2026
7 of 8 checks passed
@EiffL
EiffL deleted the docs-user-guide branch August 21, 2026 08:17
EiffL added a commit that referenced this pull request Aug 21, 2026
Final pass of the documentation rewrite — no doc pages this time, just
bringing the two things that *govern* the docs in line with their new
reality.

## CLAUDE.md

- **The freeze rule is replaced** by its successor: docs are live (PRs
#185#188), two tracks, and a change lands with its docs — a verb change
updates its `docs/cli/` page, an engine change its `docs/api/` module
page, user-visible behavior the user guide. The docs inherit the
codebase's own discipline: document only what exists, quote refusals
from real runs, verify command blocks by executing them.
- **The design-doc pointer** now records the decision taken during this
rewrite: the design records stay in the sibling checkout and are
**dropped when the rebuild completes** — never imported into `docs/`
(reverses the paragraph's earlier plan).
- **The workflow inventory** catches up: eval runs on dispatch or the
`run-eval` label (re-trigger by re-adding it), `check-docs.yml` reviews
merged PRs, `docs-deploy.yml` tracks releases; the docs build command is
stated.
- The extending table's "add the next layer" row loses "Docs are
deliberately deferred".

Deliberately **not** done: slimming CLAUDE.md's invariant sections now
that the developer corner overlaps them. CLAUDE.md is still the working
memory for the remaining rebuild (Perlmutter spike, deferred layers),
and gutting it mid-rebuild trades a known-good reference for dedup.
Worth revisiting when the rebuild closes and the design docs drop.

## check-docs.yml

The post-merge doc-drift check was written before the freeze and knew
nothing about the structure. Its prompt now carries:
- a **map from change kind to doc home** (verb → `docs/cli/`, module →
`docs/api/` + architecture, user-visible → user guide + README,
workflow/conventions → contributing);
- the two failure modes that matter most for these docs: **quoted
console output drifting** from what the CLI prints, and **documenting
anything the code no longer delivers** (the no-foreshadowing rule
applied to docs);
- an updated skip list (CLAUDE.md and `evals/` are maintained
separately) replacing the stale `skills/` rule.

## Verification

- `zensical build`: zero issues.
- check-docs.yml validates as YAML.
- Grep for `frozen`/stale workflow descriptions in CLAUDE.md comes back
clean.

This closes the 5-PR docs rewrite. The site deploys with the next
release (or a manual `docs-deploy` dispatch).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
EiffL added a commit that referenced this pull request Aug 21, 2026
…ibility (#190)

This merges the clean rebuild of lightcone-cli onto main — the full
re-architecture from the Snakemake shim to the model where **the locked
environment is the execution environment**. The pre-rebuild codebase was
deliberately stripped and functionality re-added one layer at a time,
each layer landing with its own tests, dependencies, and (since the
rewrite) docs. 21 PRs, all individually reviewed and merged into
`clean_rebuild`; this PR is the promotion.

## What lightcone-cli is now

A project is `pyproject.toml` + `uv.lock` + `astra.yaml` — uv is the
only environment substrate, and mode is derived, never configured:
direct by default (recipes in the project's `.venv` under
Landlock/Seatbelt), containerized when `[tool.lightcone.image]` is
declared (recipes in a content-addressed image archived in the
repository itself). Five verbs: `init`, `materialize`, `status`, `run`,
`build`.

- **Identity**: `definition_version` (what an output *is*) and
`env_version` (what it ran under) are separate on purpose — a spec
contradiction is `stale` and remade; an environment move is `behind`,
reported and left alone. `--refresh` widens by exactly that one state.
- **Storage**: the DataLad model — git carries history, git-annex
carries bytes, behind ordinary `git add`/`git commit`. Every output is
committed with a manifest and a run record `datalad rerun` can replay;
runs start and end on a clean tree; foreign writes are detected by
history and classify stale.
- **Execution**: one exec boundary (`Policy` → pure argv rewrite →
honest attestation) shared by recipes and the `lc run` probe; a recipe
writes its own output directory and nothing else; denials carry
copy-pasteable remedies. Network is uncontrolled on every mechanism,
symmetrically attested.
- **Venues**: detected, never configured — a SLURM allocation is spanned
with one worker per node; known centers get a login guard whose refusal
prints the center's own `salloc`/`sbatch` spellings; podman-hpc makes
multi-node containerized runs real.
- **Publication**: declaring `[project].license` turns on the RO-Crate
view — `ro-crate-metadata.json` converged by materialize, validated
against Provenance Run Crate 0.5 in CI. Deposit is `git archive`.
- **The engine is the host's uv tool, never a project dependency** — a
project's lock carries only what the analysis imports, and an engine
upgrade rebuilds nothing.

## The layers, as merged

| PR | Layer |
|---|---|
| #173 | Layer 5: the exec boundary and `lc run` (landed first — the
smallest honest consumer of the boundary) |
| #175 | Layers 2 + 4: identity, provenance, `lc materialize` |
| #178 | The engine is the host's uv tool (deletes layer 3 by decision)
|
| #180 | Layer 6: the container hatch |
| #181 | Layer 7: venues — SLURM in-allocation, podman-hpc |
| #182 | Layer 8: the publication view |
| #184 | Hardening: the recorded residues closed (UV_* scrub,
write-scope narrowing, symmetric network attestation, machine-level
uv.toml advisory) |
| #183 | The agentic eval reactivated: `uv tool install` of the branch
under test, empty-environment task, RO-Crate gate — three green runs |
| #185#189 | The documentation rewritten end to end: README, user
guide, CLI reference, developer corner, governance — every command block
executed before shipping, every refusal quoted from a real run |

## Verification

- The suite (tests, lint, mypy strict) is green on `clean_rebuild`,
including the three gated suites CI requires: sandbox enforcement
(Landlock on Linux, Seatbelt on macOS), container smoke (real build →
archive → bytes-free-clone `datalad rerun`), and crate smoke (the
official validator, REQUIRED clean).
- The agentic eval passes end-to-end on this branch: an agent builds a
supernova-cosmology analysis from the spec, manages the environment with
`uv add`, materializes through the real engine, and publishes a crate —
$0.70–$1.22 per run.
- `zensical build` renders the rewritten docs with zero issues; the site
deploys on the next release.

## Known deferred work (tracked, not blocking)

- The one-time **Perlmutter spike** (layer 7's pending items are listed
in CLAUDE.md) — HPC support ships honestly labeled "early days".
- **hub/GKE + Cloud Build** venue and **apptainer/singularity** runtimes
— design headroom recorded, deliberately not stubbed.
- Multi-arch image archives; the design-doc drop and CLAUDE.md slimming
when the rebuild formally closes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant